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toward error in any type of indirect poll. 


campaign. 


1Literary Digest, Nov. 14, 1936. 


Public Opinion. 


THE VALIDITY OF MAIL-BALLOT POLLS* 


The trend of polling practice seems now to be toward direct 
contact with the respondent, suggesting uncertainty toward the 
indirect-contact or mail-ballot plan. Mere assumption of the su- 
periority of the former leaves unexplained the fact that the Digest 
poll was singularly successful prior to 1936 while employing the 
mail-ballot exclusively. The present investigation was undertaken 
to discover, within limits of available resources, the reasons for this 
reversal with a view toward identifying factors that may contribute 


Popular explanations included a great “protest vote” in 1936 
and failure to reach certain social strata. Those were scouted by 
the poll sponsors,’ who pointed out that identical sampling methods 
were used for both the 1932 and 1936 elections. Applying Robin- 
son’s criterion of validity’ of the state-by-state average “spread” 
between plurality forecast and the actual vote, in the two elections 
of 1932 and 1936, a greater Republican overprediction was found 
in every state in 1936 and an average state error of 37% as com- 
pared to 9% in 1932. This, together with the popular guesses, 
indicated that a peculiar combination of circumstances must have 
rearranged pre-existing sources of bias inherent in the poll’s method 
so that instead of cancelling they became cumulative in 1936. 

In order to explore these sources of error two approaches were 
used. The first consisted of the enlistment of the services of a 
nation-wide interview organization. The following questions were 
submitted to a nation-wide cross-section as part of the regular poll, 
in May, 1937. The survey resulted in 848 respondents out of 
2,929 stating they had received Digest ballots during the 1936 


*Recommended for publication by Dr. J. R. Kantor, January 16, 1939. 


"Recent Developments in the Straw Poll Field,” (See Bibli 
’The authors acknowledge indebtedness to Dr. George Gallup for 
courtesy in permitting the use of facilities of the American Institute of 
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8. Did you receive a Literary Digest straw vote ballot in the Presidential 
campaign last fall? 
Yes _....Don’t remember 
Ir ANswer To Question 8 Is “Yes,” Ask 8a 


8a. Did you send it in? 


(Ir Answer To 8a Is “Yes,” Asx:) Did you change your mind regarding 
the candidate between the time you sent it in and the election? 

Yes _......Don’t remember 
(Ir Answer To 8a Is “No,” Asx:) Why didn’t you send it in? 
For which presidential candidate did you vote in the November election? 
Roosevelt _...Landon Thomas ...Lemke -—Didn’t Vote 


—AV+ —P+ —TY —Tel. —Car —Man —Wh 
—AV —P —O —No Tel. —No car —Wh —Cl 


OccupaTION Estimated Age 

STREET City 

Key: * 
OR On Relief 
P Poor 
P+ __.. Between “P” and “AV” 
es . Average Income 
AV+ Prosperous 
Tel. Telephone 
No Telephone 
WH White 
CL Colored 
TY 


Too young to vote in 
1932; will vote for Ist 

Pres. 1940 
In addition to this survey an intensive study of a representative 
community was undertaken for purposes of comparison. Cedar 
Rapids, Iowa, was chosen because its trade with a large rural area 
and its several important industries (cereals and meat-processing) 
impart both rural and urban characteristics. The Digest mailing 
list for Cedar Rapids, containing over 10,000 names of resident 
telephone renters, was obtained’ and every fifteenth person—a total 
of 693—was called by telephone. Calls were repeated until all 
available fespondents were contacted and asked the following 


questions: 
'Through the courtesy of Mr. John Hodgins of Funk and Wagnalls. 


Yes _...Don’t remember 
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A. This is of Iowa City. I am making 
a survey, in connection with my work a: the University of Iowa, to find 
out why the 1936 Literary Digest poll failed to work. One way of getting 
at this is to ask representative members of the community about their ex- 
periences. Now: 

1. “Did you happen to receive a straw ballo: card from the Literary 
Digest during the last presidential campaign?” 

(If answer is “no” or “don’t know,” skip to B and then to 4) 

2. (If answer is “yes”) “Do you remember what you did with it?” 

2a. (If not returned) “Why?” 

B. It would help a great deal in this survey if you would give me a 
little addi-ional information which will be kept in complete confidence. 
Merely for the purposes of this survey, I thee. like to know: 

e- (If ballot had been returned) “For whom did you mark your bal- 

4. “How did you vote in she 1932 presidential election?” 

6. “What did you think of the Digest poll before 1936?” (Answers 
to this question were recorded and classified under the proper a‘titudinal 
category immediately after each interview. Examples of typical responses: 
Favorable, “They’ve always been very fair, I’m sure.” “I thought it was 
a good thing.” Neutral, “It was as accurate as anything of the sort could 
be, but I reserved judgment on its always being right.” Indifferent, 
“Didn’t think anything about it at all.” “Never followed it up.” Un- 
favorable, “I thought they hand-picked the Republican voters.” “I figured 
it was propaganda; it couldn’t mean anything.”) 

. One other matter, merely for the purpose of classifying this data: 

“Would you mind giving me your approximate age?” 

“What make of car do you have? Year or model?” 

“What is your occupation?” 


These interviewees were divided into economic classes accord- 
ing to Institute criteria (page —) from information directly ob- 
tained and later checked with property values, credit guide ratings, 
and other sources. 

In this survey 554 interviews were completed, except for age, in 
all details. Of 282 persons favoring Landon, 23 failed to vote; 
and of 251 preferring Roosevelt, 22 did not vote. 


ANALYsIs OF ReporTLd RETURN OF BALLOT 


The Digest received back 23.5% of approximately ten million 
ballots mailed. Table I, however, discloses that 515 (61%) of the 
848 persons who reported to Institute interviewers as having re- 
ceived ballots were likewise certain of having returned them. This 
apparent inconsistency is explained in Table II, which shows return 
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of ballots by Cedar Rapids persons on the 1936 poll mailing list: 
59% of the Cedar Rapids respondents who were aware of receiv- 
ing ballots likewise reported returning them. This is a figure com- 
parable to the reported return of 61% mentioned above. But the 
reported return from the total number interviewed (those to whom 
ballots were actually mailed) in Cedar Rapids was 23.6%, prac- 
tically identical with that reported for the nation. The difference 
represents recall versus failure to recall. 


Bias IN THE OrIGINAL Pott 


Age bias. In the adult population those under 45 outnumber 
those over 45 by roughly 48 to 26 million, or a proportion of 65 
to 35 per cent. Table III indicates that in these samples of the 


Taste I 


ReTuRN oF BALLOT 
(Institute Survey) 


Voters in Return Non- Return No ans. Total 
1936 return uncertain 

Roosevelt 245 166 17 3 431 
Landon 239 79 14 5 337 
Thomas 2 1 3 
Lemke 3 1 1 ; 5 
Didn’t vote 20 3 1 29 
Too young 10 8 18 
No politics 14 ll 25 

Total 518 (61%) 848 (100%) 
Taste II 


Recerpt AND RETURN OF BALLOT 


(Cedar Rapids Survey) 


Preference A B Cc D EB F G 

in 1936 R-Rte R-NRt R-DKRt TotalR NoR  DKR _ Total 
Landon 92 37 #5 (134) 118 30 282 
Thomas 2 1 (3) 2 5 
Lemke 1 (1) 2 3 
No politics 2 (2) 13 


Total 131* 78 13 (222) 262 70 554 
*59% of D, 23.6% of G 


VALIDITY OF MAIL-BALLOT POLLS 


Key: 

R-Rt Received ballot and returned it 

R-NRt Received ballot, but did not return it 
R-DKRt Received ballot, but uncertain as to return 
Total R Total reporting receipt of ballot 

No R Denied receiving ballot 

DKR Uncertain as to receipt of ballot 


send-out these ratios were not present. An Institute survey during 
the campaign found 59 per cent of new voters, and a clear major- 
ity of those between the ages of 21-44, inclined toward Roosevelt; 
and an indication of pro-Landon majority among older voters. This 
trend is substantiated in Table III. 


Taste III 
Ace AND Po.iticAL PREFERENCE 
Institute survey Cedar Rapids 
Years Roosevelt Landon N 
21-24 20 (71.4%) 8 (28.6%) 4 
25-34 94 (58.7%) 66 (41.3%) 65 
35-44 127 (59.6%) 86 (40.4%) 131 
—Total under 45 years. Ges. 200 (39%) 
45-54 106 (55.5%) 85 (44.5%) 154 
55 up 53 (39.6%) 81 (60.4%) 165 
—Total over 45 years 319 (61%) 
726 (100%) 519 (100%) 


The median age in the general population is 39.6, whereas the 
median for the Cedar Rapids sample was 49. These considerations 
point to a considerable age bias in the Digest send-out. Robinson 
suggests’ that the Digest list inclines toward a criterion of wealth 
since older people tend to have more wealth than the younger. 

Sex bias. Men, according to the Digest, outnumber women 
on its mailing list by a proportion of 90 to 10. They vote in an 
approximate ratio of 60 to 40. In the Cedar Rapids survey men 
were found to outnumber women in a ratio of 84 to 16; they, more- 
over, returned a greater proportion of their ballots. Dispropor- 


‘Robinson, op. cit. 
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tionate sex distribution may not in itself have produced any con- 
siderable error in the 1936 poll, but on a question wherein there is 
a sex differential—such as prohibition—it may loom large. 

Party bias. Since it is apparent that the 1936 Digest ballots 
went chiefly to telephone renters, three classes—farmers, laborers 
and reliefers—would be more poorly represented in the poll at 
the very time they were figuring prominently in a Roosevelt plural- 
ity. Hence, a Republican bias in the send-out might be expected. 
A comparison of the official vote in Cedar Rapids, and of the 
nation, with the political preferences of ballot recipients who were 
interviewed does reveal a six per cent and a twelve per cent plural- 
ity, respectively. However, this over-Republicanism in the original 
sample is minor compared to that in the effective sample of ballots 
returned, which, averaging all states, was 37 per cent. The bulk 
of this error consequently must be accounted for by factors related 
to ballot return. 


Bras From SELECTION IN COOPERATION 


Party selection. The relative extent to which Roosevelt and 
Landon adherents cooperated in returning ballots is shown (Table 
IV) to be significant in both sets of data. Some such selection in 
cooperation in a poll has been discussed, but its actual extent had 
not hitherto been actually measured (2, 3, 4, 5). 


Taste IV 


PouiticAL PREFERENCE AND RETURN OF BALLOT 


Critical 
Total Roosevelt Landon (diff) Ratio P 
Inst. 484 245 (56.8%) 239 (71%) 3.44 4.09 100 
Return: 
C.R. 129 37 (14.7%) 92 (32.6%) 3.55 5.04 100 


With respect to non-return of ballots by different party supporters, no 
important differences appeared, i.e. in the reasons advanced. 


Attitude toward Digest poll. On the theory that attitude 
toward the poll is indicative of degree of willingness to take it 
seriously, a check was made on the 553 Cedar Rapids respondents. 


| 
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Inspection of Table V indicates that Landon supporters were more 
favorably disposed toward the poll, as were also those who re- 
turned ballots, compared to those who did not or were uncertain 
regarding return. 


V 
Attitrupe Towarp Dicest Pott 
(Cedar Rapids Survey) 
Rec’d-non-ret. Didn’t receive or 
Rec’d-ret. or uncertain ret. receipt uncertain 
Dem. Rep. Dem. Rep. Dem. Rep. 
Favor- 27 77 11 20 48 75 
able (71.9%) (83.7%) (24.4%) (47.7%) (28.2%) (50.6%) 
1 6 2 1 6 4 
Neutral (5%) (30%) (10%) (5%) (30%) (20%) 
Indif- 3 1 14 4 77 56 
ferent (5.4%) (1%) (31.1%) (16.6%) (45.3%) (37.8%) 
Unfavor- 7 8 18 14 38 13 
able (18.9%) (8.6%) (40%) (33.3%) (223%) (87%) 
Total 37 92 45 42 169 148 


(100%) (100%) (100%) (100%) (100%) (100%) 


Economic selection in cooperation. Persons in the higher in- 
come brackets have been found to be more apt to return mail bal- 
lots than persons in the lower brackets ranging from 40 per cent 
for those listed in Who’s Who to 11 per cent in relief. This is 
substantiated in the results from both surveys shown in Table IV. 


Tasie VI 
Economic STaTus AND Return oF * 
AV-+ AV P+ P 
Institute 119 (68%) ** 203 (63%) 48 (49%) 65 (55%) 


Cedar Rapids 15 (47%) 79 (29%) 31 (15%) 4 (14%) 


*Minor parties not included. 
**Number and per cent returning ballot. 
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The foregoing considerations suggest these conclusions: 

Those who had decided to vote Republican in the election tended 
to regard the poll as an opportunity to protest against the ad- 
ministration, likewise tending to regard the poll with more 
familiarity and esteem on account of their more favorable economic 
status; while those who had decided to vote Democratic in the 
election generally lacked incentive to voice their affirmation of 
the administration in a poll. 


There was a Republican bias in the 1924, 1928 and 1932 polls, 
revealed by totalling the past preferences of those who participated 
in these polls." This bias presumably arose from a consistent, 
but not high, relationship between Republicanism and higher 
economic status (represented by propertied persons with cars and 
telephones). The bias was considerably counteracted then by a 
greater proportion of return by Democrats on the mailing list, since 
theirs was the position ‘during those years of a minority party 
protesting against the party in power. 

In 1936, however, the marked increase in the relationship be- 
tween Republicanism and higher economic status made for a 
greater Republican over-representation on the mailing list during 
the very year that the Republicans shifted vantage points with 
the Democrats, becoming a minority party to whom the poll rep- 
“resented an excellent opportunity to protest against the party 
in power. 

Hence the two biases of over-representation of one party in 
the poll mailing list and disproportionate return by members of 
one party, did not serve to cancel each other as they did in 1924, 
1928 and 1932, but became cumulative in 1936, contributing more 
than any other single cause to the failure of the poll to predict 
the election outcome. Other factors of course contributed; but 
this phenomenon alone indicates that in using the mail ballot in 
gathering data on social questions, close attention must be given 
to the current relative position of the “outs” and the “ins” to avoid 


an error which may assume considerable magnitude. 


1Robinson, C. E., Straw Votes, New York, Columbia Univ. Press, 1932. 
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